To have a robot grip a moving object by itself or dock a spacecraft with another spacecraft, it is necessary to recognize and track a target mark on the moving object or the spacecraft using a video camera.
There has heretofore been known a process of measuring the position and attitude of an object by producing an image of a target mark on the object with a video camera, and processing the data of the produced image to determine the position and attitude of the object. The process may be used in an application for gripping the object with a robot hand. In such an application, the video camera is mounted on a robot, which tracks the target mark based on position and attitude data of the target mark which are produced by the video camera, for gripping the object with the robot hand.
The conventional process takes its time until the position and attitude of the object are recognized by processing the image data of the target mark. It has been impossible for the prior process to effect a real-time data feedback to the robot and also difficult to track the object.
Another process which effects pattern matching on images to track an object is time-consuming as it requires lots of calculations in a two-dimensional space.
According to still another process of tracking an object, movement of the object is grasped, and the position of the moving object is predicted. This process cannot simply be applied to movement of an ordinary object because the process is based on the fact that the object makes regular movements.
Before a target mark is recognized, it is necessary to extract a desired mark from an image which either contains another object or objects or has a lot of noises. To meet such a requirement, the conventional processes compare the area of the mark or extracts features by way of pattern matching.
The area comparison procedure determines as a desired mark an extracted image having substantially the same area as the desired mark. It is virtually impossible, however, to extract a desired mark from an image which either contains an object of almost the same size around the mark or has a lot of noises. The area comparison procedure thus finds use in a limited range of applications.
The feature extraction procedure based on pattern matching needs a large expenditure of time for searching an image memory, and hence it processing time is long.
To measure the position and attitude of an object in a three-dimensional space, there is employed a triangular or rectangular target mark representing the positional relationship between three or four points. If such a target mark is attached to a certain plane of an object, then the position and attitude of the object can be measured from the positional relationship of the image of the target mark in an image space. In the measurement, calculations based on projective geometry are effected on the coordinates of image points that are projected from the object space of the target mark onto the image plane of a camera. When the position or attitude of the object changes, the relationship between image points on the target mark also changes. Therefore, it is possible to calculate the position and attitude of the object in the three-dimensional space based on the change in the relationship between image points on the target mark.
Since a conventional measuring system using target marks calculates the position and attitude of an object based on the coordinates of image points that are extracted from a triangular or rectangular target mark image, the measuring accuracy tends to vary depending on the attitude of the target mark with respect to the camera. Specifically, when image data containing a directional component is obtained from each image point on an image plane to describe a certain plane of the object to which a target mark is attached, a reference distance with respect to each image point varies, resulting in a lack of stability with respect to the measuring accuracy for the position and attitude.
Conventional calculations of a position using a target mark require that the plane of the target mark be at a certain angle to the plane of an image, and hence need much more calculation parameters than if the camera faces the target mark head on. Therefore, the calculations in measuring the position are complex, and the measuring accuracy is lowered.
When a mark in the form of four points is converted into an image by an imaging means, the four points are shown as having a certain area on the image, making it impossible to accurately determine the positions of the points in the image data. Accordingly, the positions of the points on the image cannot be determined in terms of subpixels. Since the distance up to the object and the attitude of the object are calculated based on the inaccurate positions of the points in the image data, the distance up to the object and the attitude of the object cannot be measured with accuracy.